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APPARATUS FOR THE CHARACTERISATION OF PIGMENTED 
SKIN LESIONS 

This invention relates to an apparatus for the characterisation of 
pigmented skin lesions which is designed to assist doctors in diagnosing 
pigmented skin lesions in general, and melanomas and the like in particular. 

The apparatus comprises an instrument for the acquisition of a plurality 
5 of images of the lesions, filmed with lighting at different wavelengths, means 
designed to segment and parameterise each of said images, means designed to 
extrapolate a data set from said images and input said data into a neural 
network system, means designed to compare the results processed by said 
neural network with the results obtained following similar processing of 
10 known cases, and means designed to vary the weighting of each parameter 
supplied to the neural network on the basis of said results. 

The apparatus then supplies a parameter which allows the lesion to be 
classified under one of the categories commonly used in clinical diagnosis, 
such as "probable melanoma", "suspect case", "doubtful case" and "probable 
1 5 non-melanoma". 

The importance of early diagnosis of tumours, including epidermal 
tumours, is well known. 

In the specific case of epidermal tumours such as melanomas, early 
diagnosis is based on visual observation of a series of characteristics of the 
20 lesion, among which Asymmetry of the lesion, ragged Border, Colour and 
Dimension (A B C D) have acquired particular importance over the years. 

However, these parameters obviously require subjective evaluation by a 
doctor, which means that the result is strongly influenced by the doctor's skill 
and experience, and by incidental factors such as lighting conditions and the 
25 like. 



Modern technology provides some sophisticated instrumentation which 
allows enlarged images of the lesion to be obtained under different conditions, 
but the evaluation and consequent classification of the lesion still depend on 
the doctor's experience, and his evaluation relies on a visual impression which 
5 can vary according to the conditions of the moment. 

The modern systems available include epiluminescence microscopy, 
spectrophotometry methods and infra-red imaging. 

The characteristics of the lesion, such as ragged edges, colour and/or 
presence of darker areas, etc., are parameters that vary to a greater or lesser 
10 extent in the presence of disease, enabling the doctor to assess whether or not 
the lesion belongs to the melanoma category. 

The digital imaging technique with evaluation of reflectance under 
different lighting conditions allows determination of various parameters that 
may be typical of melanoma, such as cutaneous blood, pigmentation and the 
15 presence of melanin, which present different optical characteristics in the 
presence of disease. 

Recent computerised image-processing techniques reveal the 
morphological characteristics of the lesion and allow study of its structure, the 
conformation of the vascular network and the presence of any cell aggregates, 
20 all of which parameters are very important for the purpose of establishing the 
existence of a melanoma. 

Nevertheless, however useful these systems may be, the assistance they 
provide is limited, because they are unable to improve the doctor's evaluation 
skills or enable him to work under standardised conditions, with the result that 
25 the doctor's subjective evaluation is still based on his personal experience, and 
early diagnosis of melanoma still presents a high error rate, even when 
performed by skilled doctors. 

The present invention, which falls into this sector, relates to an 
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apparatus for the characterisation of pigmented skin lesions which is designed 
to provide doctors with information useful in classifying the lesion by 
assigning it to a type or group of types including, for example, probable non- 
melanoma, doubtful cases and probable melanoma. 
5 The apparatus according to the invention is based on the use of a neural 

network system. 

The apparatus comprises an instrument designed to acquire a set of 
images of the lesion, filmed with lighting at different wavelengths, and to 
process said images to extract the descriptors of the lesion, means designed to 
10 reduce them to a limited number of descriptors, but in such a way as to 
maintain the total data variance almost entirely, means designed to compare 
said data with a previously stored data set relating to analysis of a series of 
lesions in order to extrapolate a value indicative of a type of pathological 
state, and means designed to recalibrate the system at intervals. 
15 In order to explain the invention more clearly, the apparatus according 

to the invention and its method of operation will now be described by 
reference to the annexed drawings, wherein: 

fig. 1 is a block diagram of the system of acquisition of digital images 
with the apparatus according to the invention; 
20 fig. 2 is a block diagram of the method of operation of the apparatus for 

processing and classification of pigmented skin lesions; 

fig. 3 is a schematic representation of the model of dynamic image 
analysis performed with the apparatus according to the invention; 

fig. 4 schematically illustrates an apparatus according to the invention; 
25 fig. 5 is a diagram of the neural network also representing input values 

ih h — > H, the same values V u i V 6 normalised and output value n\ the 
threshold value is s=0. 7760; 

fig. 6 shows a dynamic lesion-classification curve. Dynamic curve 
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obtained by varying i'j between 0 and 1. The values on the x-axis are 

multiplied by 100; 

fig. 7a show dynamic curve obtained by varying i V 
fig. 7b show dynamic curve obtained by varying V 2 : 
5 fig. 7c show dynamic curve obtained by varying i' 3 ; 

fig. 7d show dynamic curve obtained by varying i 9 4 \ 
fig. 7e show dynamic curve obtained by varying i V 
fig. 7f show dynamic curve obtained by varying V 6 ; 
fig. 8 is a histogram representing risk values of lesion examined; 
10 fig. 9 is an example of dynamic curve for a lesion classified as a 

melanoma; A(NM) and A (CM) represent the areas under the dynamic curve 

and the horizontal line corresponding to threshold value s of the network; 
fig. 10a show dynamic curve obtained by varying 
fig. 10b show dynamic curve obtained by varying i' 2 ; 
15 fig. 10c show dynamic curve obtained by varying i' 3 ; 

fig. lOd show dynamic curve obtained by varying i' 4 \ 
fig. lOe show dynamic curve obtained by varying V 5 \ 
fig. 1 Of show dynamic curve obtained by varying i' 6 ; 
fig. 1 1 shows an example of distribution of determinance for the various 
20 descriptors of the lesion examined. p inc (l)=0.91 9 p inc (3)=0.85, p inc (6)=0.83 

(see figures 3a,c,f). 

As shown in figure 4, the apparatus according to the invention 

comprises a probe 1 equipped with an imaging system 2 designed to film the 

lesion, which is illuminated by a device 3 more particularly described below, 
25 said device being connected to probe 1 via a fibre optic cable 4. The probe is 

then connected via an interface 5 to a computer 6, which in turn is 

implemented with a neural network. 

The neural network may consist of hardware devices or programs in 
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which a set of elements initially has a random connection or a connection 
entered on the basis of pre-set criteria. The neural network is then taught to 
recognise a configuration by strengthening the signals that lead to the correct 
result and weakening incorrect or inefficient signals; the neural network 
5 consequently "remembers" this configuration and applies it when processing 
new data, thus giving rise to a kind of self-learning process. 

Probe 1 is installed in a body 7 with an aperture 8 shaped so that it can 
be rested on the patient's skin around the lesion to be tested. 

The lighting device comprises a light source 9, a filter 10 designed to 
10 eliminate blue and ultra-violet light, a concave mirror 15, a colour separator 
12 and an optical unit 13 for uniform distribution of light in the area of the 
lesion to be filmed. 

Lamp 9 may be a halogen lamp or a xenon lamp, for example. 

Acquisition window 8 of the probe may be fitted with a number of legs 
15 to adapt it better to the surface of the area studied and allow the probe to be 
positioned, preferably at right angles to the skin surface, in such a way as to 
ensure better acquisition of the image by the systems with which the probe is 
fitted. 

Colour separator 12 comprises a step motor 16 which causes a concave 
20 mirror 14 with a diffraction grid to rotate around its own axis. 

A second concave mirror 15, with no diffraction grid, is fitted between 
filter 10 and mirror 14 and can be rotated, by means of devices not illustrated 
in the figure, between a position shown in the figure with a broken line, in 
which it receives and reflects the illumination, and a position represented by 
25 an unbroken line in which it does not interfere with the path of the light. 

Mirror with diffraction grid 14 breaks down the light from lamp 9 into a 
series of spectral bands with pre-selected wavelengths A, 3 

Mirror 15 is moved to a position in which it acquires a colour image of 



the same area by means of a video camera 16, preferably the triple-sensor 
type, fitted with a lens 17. 

Fibre optic bundle 4, which directs the light towards the area to be 
filmed, is given a ring configuration close to the terminal end, so that the 
5 fibres are arranged all round lens 17 of the video camera and illuminate the 
filming area as uniformly as possible. 

A set of optical units 13 serves to distribute the light better. 

The imaging system comprises the video camera with a sensor 18 for 
black and white filming, which is sensitive to infrared rays, and a second 
10 sensor, or preferably a set of 3 sensors, 19, for RGB filming. 

The optical unit consisting of the lens is schematically represented by 
lens 17 and two more lenses 20 and 21; however, this is merely a schematic 
layout, and the optical unit could also be a complex type. 

Sensors 18 and 19, preferably constituted by CCD sensors, are both 
15 connected to interface 5 and, via said interface, to computer 6. 

The assembly will also advantageously comprise a calibrated light 
source, not illustrated in the figure, for calibration of the device, and in 
particular for the "blank calibration" to be performed before each set of 
images is filmed, especially before RGB filming. 
20 To film a lesion, the device is first calibrated by reading a surface lit 

with a calibrated light, so that the electronics of the device calibrate the 
reading curves of the sensors in accordance with a known technology. 

The apparatus is now ready, and images of the lesion can be obtained by 
resting the probe on the epidermis in such a way that the lesion is enclosed 
25 within reading window 8. 

Black and white readings are obtained by removing mirror 15 so that 
the light from lamp 9 is reflected by mirror with diffraction grid 14. 

Mirror 14 is rotated by motor 16 through the angle required to reflect 



light with the pre-selected wavelength; said light passes through fibre optic 
cable 14 and illuminates the area to be filmed. 

The image is filmed by lens 17, which transfers it to sensor 18; from 
there, it is conveyed via interface 5 to computer 6 for saving and subsequent 
5 processing. 

A set of images are filmed, the angle of mirror with diffraction grid 14 
being varied each time so as to vary the wavelength of the light that 
illuminates the lesion. 

In this specific case, the device will advantageously be designed to film 
10 with light at a wavelength of between 480 and 1000 nanometres. 

Mirror 15 is then rotated to reflect all the light from lamp 9, without 
limiting the wavelength band, in order to perform RGB filming with the 
second sensor 19. 

For the sake of completeness, the use of the apparatus according to the 
15 invention for characterisation of a pigmented skin lesion in the diagnosis of 
melanoma will now be described, with examples. 

A set of preliminary instructions is first supplied to the neural network, 
for example by storing data already acquired in relation to a number of lesions 
with known histological results. 
20 Information relating, for example, to dimensions, ragged edges, colour 

of the lesion, etc., is stored, and on the basis of this information the machine 
performs a first classification of new lesions using an algorithm implemented 
on the basis of said known data. 

The operation of the apparatus for objective characterisation of a new 
25 lesion is represented schematically in the block diagram in Fig. 1. 

When the instrument has been calibrated, the probe is placed on the 
lesion and a digital image thereof acquired. 

For this purpose, a first acquisition is performed in RGB format, after 



which a number of images (in this specific case 15) of the same lesion, 
illuminated by light with different wavelengths determined by suitable 
rotation of mirror with diffraction grid 14, are successively acquired and 
recorded. 

5 Imaging is performed in the field of visible and infra-red radiation, with 

separate processing of each image. 

For each black and white image, a set of parameters which are 
considered significant for the purpose of determining whether the lesion is a 
melanoma are obtained. 
10 For example, the most commonly used clinical criterion, known as "A B 

C D", can be used. 

A set of parameters, including size, variegation, reflectance in the 
visible light zone, infra-red reflectance, the presence of dark patches and the 
ratio between the area of the dark patches and the rest of the lesion are 
15 obtained from each image. 

Numerous variables are thus obtained, the number of which is reduced 
by suitable statistical analysis such as factorial analysis; a limited number of 
variables can thus be selected, e.g. three for each descriptor, which still retain 
over 85%, and preferably at least 95% of the variance. These are the variables 
20 which will be input into the neural network. 

Basically, a set of descriptors of each lesion will be extracted after 
processing of the images, and successively reduced to the number of six: {i h 
i 2 ,..., id} - These six descriptors, when input into the neural network, allow the 
lesion to be classified. This is done following a comparison between the result 
25 n output by the network and the classification threshold value s previously 
obtained by teaching the neural network and determining the connection 
weightings between the neurones. 

The values of descriptors i h i 2 , ...,/* are re-expressed in terms between 0 



and 1 in accordance with the following linear normalisation procedure. 

The minimum value and maximum value of lesions m = J,2,3 r ... 
previously acquired are selected for each descriptor irfl), i } (2), ... : 
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Example: if 100 different lesions measuring between lOmm 2 and 

1 50mm 2 have been acquired, and i 6 is the descriptor relating to the dimensions 
of the lesion, then i 6 mjn = 10 and i 6 max =150. 

Each value of descriptors //, i 2 , .... /<? is then converted into new values 
/'/, i'2, —, i'6 in accordance with the linear equations: 
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It will immediately be seen that i'i tma x=l m , i'j,mm = 0, i'zmax^l'* 



t'2,min—0,. 



10 

Example: all the dimension values cited in the previous example are re- 
expressed in accordance with the equation: 
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him) 1 
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from which it will immediately be seen that i'e.max^l and i'^min^O; all 
dimension values between the minimum and maximum values are re- 
expressed in the interval between 0 and 1 . 

A data archive containing the values of the normalised descriptors {/'/, 
i'2> > * relating to all the lesion images already acquired, the colour image / 
of each lesion and the corresponding histological classification h (NM: non- 
melanoma or CM: melanoma) is thus generated and stored in the memory. 



m=l 
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Example: for the lesion m=25, the data archive will contain: 




V,(25) i' 2 (25) i' 3 (25) I' 4 (25) i' 5 (25) V 6 (25) 
0.3285 0.2041 0.7460 0.4849 0.5364 0.3275 



1(25): 



Image 
of lesion 



h=CM 



15 Dynamic analysis of lesions 

A dynamic lesion analysis is therefore conducted to evaluate the risk 
level of each descriptor processed, ie. the extent to which a variation in each 
descriptor risks varying the classification of a lesion classed as a non- 
melanoma. If the lesion is classified as a melanoma, dynamic analysis 
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evaluates the extent to which each of the descriptors determines that 
classification. Dynamic lesion analysis also allows lesions to be classified in 
two or more classes (in this specific case there are four classes: "probable 
melanoma", "doubtful", "suspect" and "probable non-melanoma"). 
5 Example of processing of a benign lesion 

Dynamic lesion analysis is based on the following principle: five of the 
six values of the descriptors examined remain fixed in turn, while the sixth 
value is varied between the minimum and maximum values in the data archive 
0" m/n-O and /' wax =l for each descriptor), and the response n which the neural 
10 network would give for each value of the sixth simulated descriptor is 
evaluated. 

Figure 5 shows the set of values {//, /2, ie) acquired by the six 
different descriptors, which in this specific case are: 
ij\ presence of dark sub-zones 
15 i 2 : ratio between areas of dark zones and total area of lesion 

i 3 : variegation 
i 4 \ infra-red reflectance 
i 5 : reflectance in the red light zone 
i 6 : size of lesion. 

20 The quantities V u i' 2i i'6 are generated following normalisation of i h 

i 2 , U\ n is the value output by the neural network, and s is the classification 
threshold value with which it is compared. 

The diagram of the neural network shown in figure 5 also shows input 
values //, i 2 ,..., i 6 , the same values i' h i' 2 ,... s i 'e normalised, and output value 
25 n; the threshold value is s=0. 7760. 

Dynamic analysis simulating the first descriptor 

If descriptors i' 2 , i'3 , i'4, i 5 and V 6 are maintained at fixed values, it will 
be possible to see how value n varies if i'j acquires another value. Specifically, 
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1000 i'i,simui values equidistant from one another, falling between 0 and 1 5 are 
generated; each set {ii >S imui> h> h> U> U> h) is input into the neural network and 
generates an output value n. A curve of the type shown in figure 6, called a 
dynamic curve, is thus obtained, in which the values on the x-axis are multiplied 
5 by 100 and each point is generated by a different value of \\ tSimul (curve A). Circle 
C indicates the point corresponding to /'/=0.2540 ? the true value of the first 
descriptor of the lesion. As quantity n associated with i'/ is equal to 1 and 
s=0. 7760, the lesion is classified as a non-melanoma, because n>s. 

It will immediately be seen from the graph that if /'/ were between 
10 threshold a [from n>s to n<s] and threshold b [from n<s to n>s] (zone a-b), 
the value of n would be less than threshold value s and the lesion would be 
classified as a melanoma. 

Dynamic analysis simulating the other descriptors 

The process illustrated above is also applied to the other five 
15 descriptors of the lesions, and a different dynamic curve is obtained for each 
simulated descriptor. An example of these curves is shown in figures 7a to If. 
Definition of risk and risk histogram 

Two quantities, known as the index of risk in the event of increase in a 
descriptor and index of risk in the event of decrease in a descriptor, are 
20 defined on the basis of the dynamic curves obtained. These quantities indicate 
the extent to which a variation in the descriptor in question can cause a change 
in the initial assessment of non-malignancy of the lesion. 

Risk of increase p inc in descriptor d (where d in this specific case ranges 
between 1 and 6) is defined as: 

25 

Pinc(d) = \-\V d -a\ wherein i'ct<a and a is the closest value of the 

< threshold between n>s and n<s 

PmcXd) = 0 if * 'cP*® or a is non-existent 
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and risk of decrease p dec in descriptor d is defined as: 



PdeX d ) = 1 -O'rf-*) wherein and 6 is the closest value of the threshold 



PdeX d ) = 0 if ' or 6 is non-existent 
As will be seen from the definitions, the shorter the distance (arrow p in 
figures 8a 5 b 5 c, d, e and f) between the value i' d and one of the thresholds a or 
b, the greater the risk value associated with that descriptor. The risk values 
obtained can all be summarised in the histogram shown in figure 8, in which 
the values of p are re-expressed in accordance with the following simple 
formulas in order to assign a discrete risk value between 0 and 10 to each 
descriptor: 



wherein "int" means the integer of the result obtained in the square 
brackets. 

The evaluation of the risk of variation in the descriptors is of particular 
clinical interest, as it tells both doctor and patient which characteristics of the 
lesion must be most closely monitored, and the extent to which they represent 
a risk. 

Example of processing of a melanoma 

The procedure used to generate dynamic curves is the same in the case 
of lesions classified as melanomas. However, the information obtained from 
these curves is different, because the definition of risk obviously does not 
make sense for lesions classified as melanomas. Figure 9 contains an example 
of a dynamic curve for a lesion classified as a melanoma; in that figure, 
A(NM) and A(CM) represent the areas under the dynamic curve and the 
horizontal line corresponding to the threshold value s of the network. 




between n<s and n>s 



P : n Ad) = \ni[\0p inc {d)] 
p: ec (d) = mt[lQ-p dec (d)] 
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Six sample dynamic curves relating to the various descriptors are shown 
in figures 10 a, b, c, d, e and f. 

Definition of determinance and distribution of determinances 
In the case of a lesion classified as a melanoma, reference will therefore 
5 not be made to risk, but to the determinance 8(d) of each descriptor. 
Determinance indicates the extent to which the values acquired by each 
descriptor d determine the classification of malignancy of the lesion in 
question. This is expressed for each descriptor by 

S(d) = S inc (d) + S dec (d) 

10 wherein S inc (d) and 8 dec (d)^ called determinance of increase in 

descriptor d and determinance of decrease in descriptor d respectively, are 
defined as follows: 

determinance of increase in descriptor d (wherein d in this specific case 
ranges between 1 and 6): 
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^ wc (flf) = A(NM) — 1 — wherein i'd<a and a is the closest threshold value 
A(CM)\i' d ~a\ 

from n<s to n>s 
<5inc(d) = 0 if i y>a or a is non-existent 

determinance of decrease in descriptor d: 

<5jec(d) = A(<NM \ 1 — r wherein i' d >b and b is the closest threshold value 

{A(CM) \i\-b) 
from n>s to n<s 
8<iec(d) = 0 if i '^b or b is non-existent 

As will be seen, the shorter the distances between i' d and any thresholds 
a and b and the higher the ratio between areas A(NM) and A (CM), the greater 
20 the determinance value S. 
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If 



10 



15 



20 



the components of determinance 8 inc and 8 dec can be re-expressed by 

S lnc {d) 



D 



to obtain normalised determinance value S* 

d=\ d=\ d=\ 

and the distribution of these values can be displayed in the chart 
contained in figure 11, which shows an example of distribution of 
determinance for the various descriptors of the lesion in question. 

In addition to information about determinance, it is equally important 
from the clinical standpoint to assess what percentage variation in each 
descriptor would lead to a different classification of the lesion. This value is 
represented in the dynamic curves by the distance between the "true" value of 
the descriptor (circle on the curve) and the nearest thresholds b [from n>s to 
n<s] and a [from n<s to n>s]. In the example given, the values obtained are 
shown in the table below: 





increase 


decrease 


i' 2 : black area 






i' 3 : variegation 




23% 


i V red reflectance 


44% 




i V IR reflectance 




45% 


I f 6 : dimension 




23% 



As will be seen by comparing the data in the table with those contained 
in the chart in figure 1 1, the quantities differ. This clearly emerges in the case 
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of dimension; although the distance that separates it from the zone of non- 
malignant lesions is equal to that of variegation (see figures 10c and f) 5 its 
determinance is much lower. This behaviour is due to the fact that the value 
range acquired by variegation for a classification of melanoma (from b to 100) 
5 is lower than that of dimension (from b to 100). 

Classification of lesions into 4 or more different categories 
To complete the explanation given so far, dynamic analysis allows 
lesions to be assigned to two or more classes. An example of the criteria for 
classification of lesions into 4 different categories (in this specific case, non- 
10 melanoma, doubtful, suspect and melanoma) is set out below. 

A lesion originally classified as non-melanoma by the neural network is 
classified as doubtful if the risk of increase or decrease in at least two 
descriptors is greater than 0.8. As these descriptors are close to the threshold, 
despite the classification of non-malignancy supplied by the neural network it 
15 is preferable to take a cautious attitude and assign the lesion to a category 
other than "non-melanoma". Similarly, a lesion originally classified as 
melanoma by the neural network is assigned to the category of suspect lesions 
if, when the dynamic curves are observed, the distance between the point 
corresponding to the lesion and the threshold is less than 0.2 for at least two 
20 descriptors. 

Example: according to this classification criterion, the non-malignant 
lesion referred to above would be classified as doubtful, because p inc (l)^0.91 , 
Pinc(3) = 0.85, and p inc (6)=0.83 (see figures 7a, c and f). 

Comparison between new lesions and data archive 
25 As in the case of any classification model, the results supplied by the 

neural network depend on the cases used to teach the network. The fewer the 
lesions belonging to the teaching cases (stored in the data archive), the greater 
the probability that a new lesion will not be "similar" to any of the previously 
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acquired lesions. 

The similarity s between a lesion m present in the data archive 
(characterised by the set of descriptors {//, i 2 ,..., i 6 }) and a new lesion 
({/*/, «.,/**}) can be quantified with the equation 




Of all the lesions in the data archive, the lesion m "most similar" to the 
one acquired is characterised by the lowest s value, which will be indicated as 
s 

10 s* - m\v\ m \s{pi^ 

At this stage a quantity a, called the lesion atypical ity index, can be 
defined with the equation 

a = e N 

wherein cr is a pre-set threshold value and N corresponds to the number 
15 of lesions present in the data archive. The smaller the value of a, the greater 
the similarity between lesion m and the new lesion. If a>l, the message 
"Warning,, no similar lesions in data archive" will be displayed at the end of 
processing of the lesion, and the classification given must be taken with a 
greater degree of caution. The value shown by the atypicality index is still of 
20 clinical interest, however, precisely because it characterises the atypicality of 
the lesion in question. 

Study of the similarity of lesions can lead to a different classification 
criterion or a refinement of the criterion previously described. In the former 
case it may be decided, for example, that if there is at least one melanoma 
25 among the ten lesions "most similar" to the new lesion acquired, the new 
lesion should be classified as a melanoma regardless of the response given by 
the neural network. 
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If it is wished to refine the classification obtained with the neural 
networks, the two classification criteria can be combined as shown in the table 
below. 



Response of neural 


Similarity to melanoma 


CLASSIFICATION 


network 






Probable non-melanoma 


no 


Probable non-melanoma 


Probable non-melanoma 


yes 


Doubtful 


Doubtful 


no 


Doubtful 


Doubtful 


yes 


Suspect 


Suspect 


no 


Suspect 


Suspect 


yes 


Probable melanoma 


Probable melanoma 


no 


Probable melanoma 


Probable melanoma 


yes 


Probable melanoma 



5 



